Search results for "test suite"
showing 7 items of 7 documents
A Visualizable Test Problem Generator for Many-Objective Optimization
2022
Visualizing the search behavior of a series of points or populations in their native domain is critical in understanding biases and attractors in an optimization process. Distancebased many-objective optimization test problems have been developed to facilitate visualization of search behavior in a two-dimensional design space with arbitrarily many objective functions. Previous works have proposed a few commonly seen problem characteristics into this problem framework, such as the definition of disconnected Pareto sets and dominance resistant regions of the design space. The authors’ previous work has advanced this research further by providing a problem generator to automatically create use…
A Feature Rich Distance-Based Many-Objective Visualisable Test Problem Generator
2019
In optimiser analysis and design it is informative to visualise how a search point/population moves through the design space over time. Visualisable distance-based many-objective optimisation problems have been developed whose design space is in two-dimensions with arbitrarily many objective dimensions. Previous work has shown how disconnected Pareto sets may be formed, how problems can be projected to and from arbitrarily many design dimensions, and how dominance resistant regions of design space may be defined. Most recently, a test suite has been proposed using distances to lines rather than points. However, active use of visualisable problems has been limited. This may be because the ty…
A comprehensive study of automatic program repair on the QuixBugs benchmark
2021
Abstract Automatic program repair papers tend to repeatedly use the same benchmarks. This poses a threat to the external validity of the findings of the program repair research community. In this paper, we perform an empirical study of automatic repair on a benchmark of bugs called QuixBugs, which has been little studied. In this paper, (1) We report on the characteristics of QuixBugs; (2) We study the effectiveness of 10 program repair tools on it; (3) We apply three patch correctness assessment techniques to comprehensively study the presence of overfitting patches in QuixBugs. Our key results are: (1) 16/40 buggy programs in QuixBugs can be repaired with at least a test suite adequate pa…
High-accuracy extrapolated ab initio thermochemistry. II. Minor improvements to the protocol and a vital simplification
2006
The recently developed high-accuracy extrapolated ab initio thermochemistry method for theoretical thermochemistry, which is intimately related to other high-precision protocols such as the Weizmann-3 and focal-point approaches, is revisited. Some minor improvements in theoretical rigor are introduced which do not lead to any significant additional computational overhead, but are shown to have a negligible overall effect on the accuracy. In addition, the method is extended to completely treat electron correlation effects up to pentuple excitations. The use of an approximate treatment of quadruple and pentuple excitations is suggested; the former as a pragmatic approximation for standard cas…
Using partial-orders for detecting faults in concurrent systems
1998
The paper suggests test derivation approaches to obtain test suites for concurrent systems based on the concept of fault coverage criteria in opposition to structural test coverage criteria. Using a partial-order model, called Mazurkiewicz Trace Machine (MTM), for test derivation, the state explosion problem can be alleviated. The derived test suites are characterized by their small size compared to test suites from traditional test derivation approaches and exhibit a defined degree of fault coverage according to certain fault models. The fault models of concurrent systems considered in the paper are based on the most common faults, acceptance, refusal, and transfer faults. A scenario of te…
Paper-based vs computer-based exams in CS1
2016
In this study, we examine the "test mode effect" in CS1 exam using the Rainfall problem. The participants started working with pen and paper, after which they had access to a computer, and they could rework their solution with a help of a test suite developed by the authors. In the computer- based phase many students were able to fix the errors that they had committed during the paper-based phase. These errors included well-known corner cases, such as empty array or division by zero.
Towards Model-Based Reinforcement Learning for Industry-Near Environments
2019
Deep reinforcement learning has over the past few years shown great potential in learning near-optimal control in complex simulated environments with little visible information. Rainbow (Q-Learning) and PPO (Policy Optimisation) have shown outstanding performance in a variety of tasks, including Atari 2600, MuJoCo, and Roboschool test suite. Although these algorithms are fundamentally different, both suffer from high variance, low sample efficiency, and hyperparameter sensitivity that, in practice, make these algorithms a no-go for critical operations in the industry.